<!DOCTYPE html>
<html lang="zh" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<meta name="description" content="book">
<meta name="generator" content="Paradox, paradox-material-theme=0.6.0, mkdocs-material=3.0.3">

<meta name="lang:clipboard.copy" content="Copy to clipboard">
<meta name="lang:clipboard.copied" content="Copied to clipboard">
<meta name="lang:search.language" content="">
<meta name="lang:search.pipeline.stopwords" content="true">
<meta name="lang:search.pipeline.trimmer" content="true">
<meta name="lang:search.result.none" content="No matching documents">
<meta name="lang:search.result.one" content="1 matching document">
<meta name="lang:search.result.other" content="# matching documents">
<meta name="lang:search.tokenizer" content="[\s\-]+">


<meta name="description" content="book">
<link rel="shortcut icon" href="../assets/images/favicon.png">
<title>实战：大文件断点上传、下载和秒传 · Scala Web 开发——基于Akka HTTP</title>
<link rel="stylesheet" href="../assets/stylesheets/application.451f80e5.css">
<link rel="stylesheet" href="../assets/stylesheets/application-palette.22915126.css">
<meta name="theme-color" content="#009688" />
<link rel="stylesheet" href="../lib/material__tabs/dist/mdc.tabs.min.css">
<link rel="stylesheet" href="../lib/prettify/prettify.css">
<script src="../assets/javascripts/modernizr.1aa3b519.js"></script>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,400i,700|Roboto+Mono">
<style>
body,input{font-family:"Roboto","Helvetica Neue",Helvetica,Arial,sans-serif}
code,kbd,pre{font-family:"Roboto Mono","Courier New",Courier,monospace}
</style>
<link rel="stylesheet" href="../assets/fonts/font-awesome.css">
<link rel="stylesheet" href="../assets/fonts/material-icons.css">
<link rel="stylesheet" href="../assets/stylesheets/paradox-material-theme.css">
</head>
<body
data-md-color-primary="teal"
data-md-color-accent="indigo"
>
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" data-md-component="overlay" for="__drawer"></label>
<header class="md-header" data-md-component="header">
<nav class="md-header-nav md-grid">
<div class="md-flex">
<div class="md-flex__cell md-flex__cell--shrink">
<a href="../index.html" title="Scala Web 开发——基于Akka HTTP" class="md-header-nav__button md-logo">
<i class="md-icon">local_library</i>
</a>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--menu md-header-nav__button" for="__drawer"></label>
</div>
<div class="md-flex__cell md-flex__cell--stretch">
<div class="md-flex__ellipsis md-header-nav__title" data-md-component="title">
<span class="md-header-nav__topic">
Scala Web 开发——基于Akka HTTP
</span>
<span class="md-header-nav__topic">
实战：大文件断点上传、下载和秒传
</span>
</div>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<label class="md-icon md-icon--search md-header-nav__button" for="__search"></label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="query" data-md-state="active">
<label class="md-icon md-search__icon" for="__search"></label>
<button type="reset" class="md-icon md-search__icon" data-md-component="reset" tabindex="-1">&#xE5CD;</button>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="result">
<div class="md-search-result__meta">
Type to start searching
</div>
<ol class="md-search-result__list"></ol>
</div>
</div>
</div>
</div>
</div>

</div>
<div class="md-flex__cell md-flex__cell--shrink">
<div class="md-header-nav__source">
<a href="https://github.com/yangbajing/scala-web-development"
title="Go to repository"
class="md-source"
data-md-source="github">
<div class="md-source__icon">
<i class="fa fa-github"></i>
</div>
<div class="md-source__repository">
yangbajing/scala-web-development
</div>
</a>

</div>
</div>
</div>
</nav>
</header>

<div class="md-container">
<main class="md-main">
<div class="md-main__inner md-grid" data-md-component="container">
<div class="md-sidebar md-sidebar--primary" data-md-component="navigation">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" data-md-level="0" style="visibility: hidden">
<label class="md-nav__title md-nav__title--site" for="drawer">
<a href="../index.html" title="Scala Web 开发——基于Akka HTTP" class="md-nav__button md-logo">
<span class="md-nav__button md-logo">
<i class="md-icon">local_library</i>
</a>
<a href="../index.html" title="Scala Web 开发——基于Akka HTTP">
Scala Web 开发——基于Akka HTTP
</a>
</label>
<div class="md-nav__source">
<a href="https://github.com/yangbajing/scala-web-development"
title="Go to repository"
class="md-source"
data-md-source="github">
<div class="md-source__icon">
<i class="fa fa-github"></i>
</div>
<div class="md-source__repository">
yangbajing/scala-web-development
</div>
</a>

</div>
<ul>
  <li><a href="../preface.html" class="page">前言</a></li>
  <li><a href="../env/index.html" class="page">Scala 环境配置</a>
  <ul>
    <li><a href="../env/env.1.html" class="page">Sbt</a></li>
    <li><a href="../env/env.2.html" class="page">IDE开发工具</a></li>
    <li><a href="../env/env.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../scala/index.html" class="page">Scala 语言基础</a>
  <ul>
    <li><a href="../scala/scala.0.html" class="page">REPL</a></li>
    <li><a href="../scala/scala.1.html" class="page">你好，Scala</a></li>
    <li><a href="../scala/scala.2.html" class="page">Scala基础</a></li>
    <li><a href="../scala/scala.3.html" class="page">流程和函数</a></li>
    <li><a href="../scala/scala.4.html" class="page">集合</a></li>
    <li><a href="../scala/scala.5.html" class="page">class和object</a></li>
    <li><a href="../scala/scala.6.html" class="page">函数式</a></li>
    <li><a href="../scala/scala.7.html" class="page">Trait</a></li>
    <li><a href="../scala/scala.8.html" class="page">并发</a></li>
    <li><a href="../scala/scala.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../basic/index.html" class="page">Akka HTTP 基础</a>
  <ul>
    <li><a href="../basic/basic.0.html" class="page">Akka HTTP 基础</a></li>
    <li><a href="../basic/basic.1.html" class="page">Web 工作方式</a></li>
    <li><a href="../basic/basic.2.html" class="page">使用 Akka Http 搭建一个简单的 Web 服务</a></li>
    <li><a href="../basic/basic.3.html" class="page">Akka HTTP 的通用抽象</a></li>
    <li><a href="../basic/basic.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../server-api/index.html" class="page">服务端API</a>
  <ul>
    <li><a href="../server-api/work.html" class="page">Akka HTTP 如何使得 Web 工作</a></li>
    <li><a href="../server-api/advanced.html" class="page">高级服务端 API</a></li>
  </ul></li>
  <li><a href="../routing-dsl/index.html" class="page">路由DSL</a>
  <ul>
    <li><a href="../routing-dsl/route.html" class="page">Route 路由</a></li>
    <li><a href="../routing-dsl/directive.html" class="page">Directive 指令</a></li>
    <li><a href="../routing-dsl/custom-directive.html" class="page">自定义指令</a></li>
    <li><a href="../routing-dsl/rejections.html" class="page">拒绝 rejections</a></li>
    <li><a href="../routing-dsl/exception.html" class="page">异常处理</a></li>
    <li><a href="../routing-dsl/file-upload.html" class="active page">实战：大文件断点上传、下载和秒传</a></li>
  </ul></li>
  <li><a href="../directives/index.html" class="page">常用指令</a>
  <ul>
    <li><a href="../directives/path.html" class="page">PathDirectives（路径指令）</a></li>
    <li><a href="../directives/method.html" class="page">directives/method.html</a></li>
    <li><a href="../directives/parameter_form.html" class="page">directives/parameter_form.html</a></li>
    <li><a href="../directives/marshalling.html" class="page">directives/marshalling.html</a></li>
    <li><a href="../directives/file.html" class="page">directives/file.html</a></li>
    <li><a href="../directives/cookie.html" class="page">directives/cookie.html</a></li>
  </ul></li>
  <li><a href="../data/index.html" class="page">数据</a>
  <ul>
    <li><a href="../data/data.0.html" class="page">数据</a></li>
    <li><a href="../data/data.1.html" class="page">JSON</a></li>
    <li><a href="../data/data.ant-design-pro.html" class="page">实战：为Ant Design Pro提供后端接口</a></li>
    <li><a href="../data/data.kryo.html" class="page">Kryo</a></li>
    <li><a href="../data/data.2.html" class="page">Protobuf</a></li>
    <li><a href="../data/data.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../test/index.html" class="page">测试</a>
  <ul>
    <li><a href="../test/test.0.html" class="page">测试</a></li>
    <li><a href="../test/test.1.html" class="page">Scalatest</a></li>
    <li><a href="../test/test.2.html" class="page">测试异步代码</a></li>
    <li><a href="../test/test.3.html" class="page">端到端测试Route</a></li>
    <li><a href="../test/test.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../actor/index.html" class="page">Akka Actor</a>
  <ul>
    <li><a href="../actor/actor.html" class="page">Akka Typed Actor</a></li>
    <li><a href="../actor/actor-test.html" class="page">Akka Actor 测试</a></li>
    <li><a href="../actor/actor.z.html" class="page">Actor小结</a></li>
  </ul></li>
  <li><a href="../oauth/index.html" class="page">实战：实现OAuth 2服务</a>
  <ul>
    <li><a href="../oauth/oauth.0.html" class="page">实战：OAuth 2 服务</a></li>
    <li><a href="../oauth/oauth.1.html" class="page">OAuth 2简介</a></li>
    <li><a href="../oauth/oauth.2.html" class="page">OAuth 2接口设计</a></li>
    <li><a href="../oauth/oauth.3.html" class="page">OAuth 2服务实现</a></li>
    <li><a href="../oauth/oauth.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../database/index.html" class="page">访问数据库</a>
  <ul>
    <li><a href="../database/database.0.html" class="page">访问数据库</a></li>
    <li><a href="../database/database.1.html" class="page">使用 JDBC 访问 PostgreSQL</a></li>
    <li><a href="../database/database.2.html" class="page">使用 Slick 访问数据库</a></li>
    <li><a href="../database/database.3.html" class="page">访问 Cassandra 数据库</a></li>
    <li><a href="../database/database.4.html" class="page">访问 Redis</a></li>
    <li><a href="../database/database.5.html" class="page">访问 Elasticsearch</a></li>
    <li><a href="../database/database.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../engineering/index.html" class="page">工程化</a>
  <ul>
    <li><a href="../engineering/swagger.html" class="page">使用Swagger编写API文档</a></li>
    <li><a href="../engineering/guice.html" class="page">使用Guice管理类依赖</a></li>
  </ul></li>
  <li><a href="../grpc/index.html" class="page">Akka gRPC</a>
  <ul>
    <li><a href="../grpc/grpc.html" class="page">gRPC服务</a></li>
    <li><a href="../grpc/build-tool.html" class="page">构建工具</a></li>
    <li><a href="../grpc/deployment.html" class="page">部署</a></li>
    <li><a href="../grpc/grpc.z.html" class="page">小结</a></li>
  </ul></li>
  <li><a href="../config-discovery/index.html" class="page">实战：配置管理、服务发现系统</a></li>
  <li><a href="../appendix/index.html" class="page">附录</a>
  <ul>
    <li><a href="../appendix/appendix.0.html" class="page">参考资料</a></li>
    <li><a href="../appendix/appendix.1.html" class="page">专业术语</a></li>
    <li><a href="../appendix/appendix.2.html" class="page">词汇表</a></li>
  </ul></li>
  <li><a href="../donate.html" class="page">赞助</a></li>
</ul>
<nav class="md-nav md-nav--secondary">
<label class="md-nav__title" for="__toc">Table of contents</label>
<ul>
  <li><a href="../routing-dsl/file-upload.html#实战-大文件断点上传-下载和秒传" class="header">实战：大文件断点上传、下载和秒传</a>
  <ul>
    <li><a href="../routing-dsl/file-upload.html#断点下载" class="header">断点下载</a></li>
    <li><a href="../routing-dsl/file-upload.html#断点上传" class="header">断点上传</a></li>
    <li><a href="../routing-dsl/file-upload.html#秒传" class="header">秒传</a></li>
    <li><a href="../routing-dsl/file-upload.html#运行示例" class="header">运行示例</a></li>
    <li><a href="../routing-dsl/file-upload.html#小结" class="header">小结</a></li>
  </ul></li>
</ul>
</nav>

</nav>
<ul style="display: none">
<li class="md-nav__item md-version" id="project.version">
<label class="md-nav__link" for="__version">
<i class="md-icon" title="Version">label_outline</i> 1.0.0
</label>
</li>
</ul>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="toc">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary">
<label class="md-nav__title" for="__toc">Table of contents</label>
<ul>
  <li><a href="../routing-dsl/file-upload.html#实战-大文件断点上传-下载和秒传" class="header">实战：大文件断点上传、下载和秒传</a>
  <ul>
    <li><a href="../routing-dsl/file-upload.html#断点下载" class="header">断点下载</a></li>
    <li><a href="../routing-dsl/file-upload.html#断点上传" class="header">断点上传</a></li>
    <li><a href="../routing-dsl/file-upload.html#秒传" class="header">秒传</a></li>
    <li><a href="../routing-dsl/file-upload.html#运行示例" class="header">运行示例</a></li>
    <li><a href="../routing-dsl/file-upload.html#小结" class="header">小结</a></li>
  </ul></li>
</ul>
</nav>

</div>
</div>
</div>
<div class="md-content">
<article class="md-content__inner md-typeset">
<div class="md-content__searchable">
<h1><a href="#实战-大文件断点上传-下载和秒传" name="实战-大文件断点上传-下载和秒传" class="anchor"><span class="anchor-link"></span></a>实战：大文件断点上传、下载和秒传</h1>
<p>本章，使用Akka HTTP和Akka Stream做为后端服务，可以很优雅的实现大文件的断点续传。原理其实非常的简单， 前端计算文件的hash（使用sha256），将hash传到后端查询是否有相同文件已上传，若有将返回已上传文件及文件长度（bytes）。 这时候前端就可以知道文件的上传进度，进而判断还需要断点续传的偏移量或者已上传完成（这就是秒传）。</p>
<p>这里有一个设计取舍：客户端对单个文件不做分片，从文件头开始上传。这样的一个好处是可简化服务端的实现， 同时也可以优化服务端对文件的存储 <em>（同一个文件将一直使用APPEND的方式写入文件，这样可以更高效的利用磁盘IO。同时， 不需要分块合并，若文件很大，生成的大量分块在文件上传完成后再次合并会是一个非常大的资源开销）</em> 。</p>
<h2><a href="#断点下载" name="断点下载" class="anchor"><span class="anchor-link"></span></a>断点下载</h2>
<p>这个怎么说呢？断点下载Akka HTTP原生支持。你只需要使用如下代码：</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/controller/FileRoute.scala#L37-L40" target="_blank" title="Go to snippet source"></a><code class="language-scala">// 支持断点续传
private def downloadRoute: Route = path(&quot;download&quot; / Segment) { hash =&gt;
  getFromFile(FileUtils.getLocalPath(hash).toFile)
}</code></pre>
<p><code>FileUtils.getLocalPath(hash)</code>函数通过对<code>hash</code>值（sha256hex）进行计算和拼接， 获取实际文件的本地存储路径再交给Akka HTTP提供的<code>getFromFile</code>指令，剩下的工作就交给Akka。</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/util/FileUtils.scala#L32" target="_blank" title="Go to snippet source"></a><code class="language-scala">def getLocalPath(hash: String): Path = Paths.get(LOCAL_PATH, hash.take(2), hash)</code></pre>
<p>我们可以通过向Akka HTTP发起<code>HEAD</code>请求来查看支持的HTTP功能，看到在反回的header里有<code>Accept-Ranges: bytes</code>， 意思是服务端支持使用字节为单位的范围下载（断点下载功能既基于此实现）。</p>
<pre><code>$ curl --head http://localhost:33333/file/download/7d0559e2f7bf42f0c2becc7fbf91b20ca2e7ec373c941fca21314169de9c7ef4
HTTP/1.1 200 OK
Last-Modified: Fri, 28 Dec 2018 14:12:32 GMT
ETag: &quot;132766a7f528d080&quot;
Accept-Ranges: bytes
Server: akka-http/10.1.6
Date: Sat, 29 Dec 2018 02:17:41 GMT
Content-Type: application/octet-stream
Content-Length: 65463496
</code></pre><div class="callout note "><div class="callout-title">Note</div>
<p>通过以下sbt task可以启动文件上传示例程序：</p>
<pre class="prettyprint"><code class="language-sbtshell">sbt &quot;foundation/runMain fileupload.Main&quot; 
</code></pre>
<p>看到如下输出代表程序启动成功：</p>
<pre><code>......
[info] Done packaging.
[info] Running fileupload.Main 
fileupload.Main$ - startup success, ServerBinding(/127.0.0.1:33333)
</code></pre></div>
<h2><a href="#断点上传" name="断点上传" class="anchor"><span class="anchor-link"></span></a>断点上传</h2>
<p>很遗憾，Akka HTTP默认不支持断点上传，这需要自行实现。但是，Akka HTTP做为一个toolkit，足够灵活且强大，实现断点上传功能so easy。</p>
<h3><a href="#断点上传实现" name="断点上传实现" class="anchor"><span class="anchor-link"></span></a>断点上传实现</h3>
<p>基于常规的代码设计方式，我们需要<code>Controller</code>、<code>Service</code>，那就先从<code>Controller</code>开始：</p>
<h4><a href="#fileroute-uploadroute" name="fileroute-uploadroute" class="anchor"><span class="anchor-link"></span></a>FileRoute#uploadRoute</h4>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/controller/FileRoute.scala#L22-L33" target="_blank" title="Go to snippet source"></a><code class="language-scala">private def uploadRoute: Route = path(&quot;upload&quot;) {
  post {
    withoutSizeLimit {
      entity(as[Multipart.FormData]) { formData =&gt;
        onSuccess(fileService.handleUpload(formData)) { results =&gt;
          import helloscala.http.JacksonSupport._
          complete(results)
        }
      }
    }
  }
}</code></pre>
<p>这里需要注意的一个指令是<code>withoutSizeLimit</code>，默认Akka HTTP对请求大小限制比较低，我们可以通过<code>withoutSizeLimit</code> 指令取消对单个API的请求大小限制，同时又不影响整个Web服务的大小限制。另外，这里通过<code>entity(as[Multipart.FormData])</code>以 <strong>Unmarshaller</strong>的方式获取整个<code>Multipart.FormData</code>对象并传入<code>FileService#handleUpload</code>函数进行处理。</p>
<h4><a href="#fileservice-handleupload" name="fileservice-handleupload" class="anchor"><span class="anchor-link"></span></a>FileService#handleUpload</h4>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/service/FileServiceImpl.scala#L30-L40" target="_blank" title="Go to snippet source"></a><code class="language-scala">override def handleUpload(formData: Multipart.FormData): Future[Seq[FileBO]] = {
  formData.parts
  //      .groupBy(Constants.FILE_PART_MAX, part =&gt; part.name.split(&#39;.&#39;).head)
  //      .async
  //      .foldAsync[FileInfo](FileInfo.Empty)((fileInfo, part) =&gt; mergeBodyPart(fileInfo, part))
  //      .mergeSubstreams
    .map(part =&gt; FileInfo(part))
    .log(&quot;fileInfo&quot;, info =&gt; logger.debug(s&quot;fileInfo: $info&quot;))
    .mapAsync(Constants.FILE_PART_MAX)(processFile)
    .runWith(Sink.seq)
}</code></pre>
<p><code>formData.parts</code>是一个Akka Stream流，类型签名为<code>Source[Multipart.FormData.BodyPart, Any]</code>。有关Akka Stream 更详细的资料请参阅<a href="https://doc.akka.io/docs/akka/current/stream/index.html?language=scala">Akka Streams官方文档</a>。这里， 每个<code>part</code>都代表<code>FormData</code>的一个字段（对应HTML 5的<code>FormData</code>类型，同时前端需要使用 <code>Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryrP4DAyu31ilqEWmz</code>方式发起请求）。每个<code>part.name</code> 都是用英号逗号分隔的三部分来做为请求的字段名，分别是：<code>&lt;hash&gt;.&lt;content length&gt;.&lt;start position&gt;</code>， 这样我们就可以在不加入任何其它字段的情况下告知服务端当前上传文件的sha256hex计算出的hash值、 文件大小（bytes）和上传起始偏移量。</p>
<h4><a href="#fileutils-uploadfile" name="fileutils-uploadfile" class="anchor"><span class="anchor-link"></span></a>FileUtils#uploadFile</h4>
<p>文件上传的核心逻辑在<code>FileUtils#uploadFile</code>函数：</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/util/FileUtils.scala#L47-L60" target="_blank" title="Go to snippet source"></a><code class="language-scala">def uploadFile(fileInfo: FileInfo)(
    implicit mat: Materializer,
    ec: ExecutionContext): Future[FileBO] = {
  // TODO 需要校验上传完成文件的hash值与提交hash值是否匹配？
  val maybeMeta = fileInfo.hash.flatMap(FileUtils.getFileMeta)
  val beContinue = maybeMeta.isDefined &amp;&amp; fileInfo.startPosition &gt; 0L
  val f =
    if (beContinue) uploadContinue(fileInfo, maybeMeta.get)
    else uploadNewFile(fileInfo)
  f.andThen {
    case tryValue =&gt;
      logger.debug(s&quot;文件上传完成：$tryValue&quot;)
  }
}</code></pre>
<p><code>uploadFile</code>函数根据是否为续传来分别调用<code>uploadContinue</code>或<code>uploadNewFile</code>函数。首先来看看新文件上传时的代码逻辑：</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/util/FileUtils.scala#L85-L117" target="_blank" title="Go to snippet source"></a><code class="language-scala">private def uploadNewFile(
    fileInfo: FileInfo)(implicit mat: Materializer, ec: ExecutionContext) = {
  val bodyPart = fileInfo.bodyPart
  val tmpPath = fileInfo.hash // (1)
    .map(h =&gt; FileUtils.getLocalPath(h))
    .getOrElse(Files
      .createTempFile(FileUtils.TMP_DIR, bodyPart.filename.getOrElse(&quot;&quot;), &quot;&quot;))
  val sha = MessageDigest.getInstance(&quot;SHA-256&quot;)
  logger.debug(s&quot;新文件，路径：$tmpPath&quot;)
  bodyPart.entity.dataBytes
    .map { byteString =&gt;
      byteString.asByteBuffers.foreach(sha.update) // (2)
      byteString
    }
    .runWith(FileIO.toPath(tmpPath)) // (3)
    .map { ioResult =&gt;
      val computedHash = Utils.bytesToHex(sha.digest()) // (4)
      fileInfo.hash.foreach { h =&gt;
        require(h == computedHash, s&quot;前端上传hash与服务端计算hash值不匹配，$h != $computedHash&quot;)
      }
      val localPath = fileInfo.hash match { // (5)
        case Some(_) =&gt; tmpPath
        case _       =&gt; move(computedHash, tmpPath, ioResult.count)
      }
      FileBO(
        fileInfo.hash,
        Some(computedHash),
        localPath,
        ioResult.count,
        bodyPart.filename,
        bodyPart.headers)
    }
}</code></pre>
<ol>
  <li>根据前端是否上传了有效的hash值（sha256hex）来判断是把文件先写入临时文件还是直接写入实际的本地存储位置（根据hash值计算出本地实际的存储位置）。</li>
  <li>Akka HTTP中，上传的文件以流的方式进入，在此对每个<code>ByteString</code>计算并更新sha256值。</li>
  <li>在Akka Stream的Sink端，接收流传入的元素并写入本地文件。</li>
  <li>文件写入结束后调用<code>sha.digest()</code>方法获取已上传文件的sha256值。</li>
  <li>根据是否临时文件来判断是否需要将临时文件移动到实际的本地存储路径，通过文件的hash值来计算出实际的本地存储路径。</li>
</ol>
<p>断点上传时的逻辑其实相对简单，需要注意的是在<code>(1)</code>处调用<code>FileIO.toPath</code>将流定入本地时需要以<code>APPEND</code>模式追加写入到已存在文件。</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/util/FileUtils.scala#L64-L81" target="_blank" title="Go to snippet source"></a><code class="language-scala">private def uploadContinue(fileInfo: FileInfo, meta: FileMeta)(
    implicit mat: Materializer,
    ec: ExecutionContext) = {
  val bodyPart = fileInfo.bodyPart
  val localPath = FileUtils.getLocalPath(fileInfo.hash.get)
  logger.debug(s&quot;断点续传，startPosition：${fileInfo.startPosition}，路径：$localPath&quot;)
  bodyPart.entity.dataBytes
    .runWith(FileIO.toPath(localPath, Set(APPEND), fileInfo.startPosition))
    .map(
      ioResult =&gt;
        FileBO(
          fileInfo.hash,
          None,
          localPath,
          meta.size + ioResult.count,
          bodyPart.filename,
          bodyPart.headers))
}</code></pre>
<h2><a href="#秒传" name="秒传" class="anchor"><span class="anchor-link"></span></a>秒传</h2>
<p>在已实现断点上传功能之上，秒传的实现逻辑就非常清晰了。客户端在调用<code>file/upload</code>接口上传文件之前先调用<code>/file/progress/{hash}</code>接口判断相同hash值文件的上传情况，再决定下一步处理。</p>
<ol>
  <li>客户端计算文件hash，并以文件hash和文件大小作为参数调用<code>/file/progress/{hash}</code>接口</li>
  <li>服务端根据上传hash值判断文件是否已上传，若存在返回已上传文档大小（bytes）</li>
  <li>客户端收到服务端响应后根据文件是否存在及已存在文件大小判断**秒传**、**断点续传**还是**新上传**</li>
  <li><strong>秒传</strong>，返回文件长度与当前准备上传文件长度大小一致</li>
  <li><strong>断点续传</strong>，返回文件大小比当前准备上传文件长度小</li>
  <li><strong>新上传</strong>，返回文件不存在</li>
  <li>其它情况，作为新文件上传</li>
</ol>
<p>秒传的代码逻辑台下：</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/controller/FileRoute.scala#L22-L33" target="_blank" title="Go to snippet source"></a><code class="language-scala">private def uploadRoute: Route = path(&quot;upload&quot;) {
  post {
    withoutSizeLimit {
      entity(as[Multipart.FormData]) { formData =&gt;
        onSuccess(fileService.handleUpload(formData)) { results =&gt;
          import helloscala.http.JacksonSupport._
          complete(results)
        }
      }
    }
  }
}</code></pre>
<p>文件上传进度服务实现如下。</p>
<pre class="prettyprint"><a class="icon go-to-source" href="https://github.com/yangbajing/scala-web-development/tree/master/foundation/src/main/scala/fileupload/service/FileServiceImpl.scala#L23-L26" target="_blank" title="Go to snippet source"></a><code class="language-scala">override def progressByHash(hash: String): Future[Option[FileMeta]] = {
  require(Objects.nonNull(hash) &amp;&amp; hash.nonEmpty, &quot;hash 不能为空。&quot;)
  Future.successful(FileUtils.getFileMeta(hash))
}</code></pre>
<h2><a href="#运行示例" name="运行示例" class="anchor"><span class="anchor-link"></span></a>运行示例</h2>
<p>通过以下sbt task可以启动文件上传示例程序：<code>sbt &quot;foundation/runMain fileupload.Main&quot;</code>。启动示例程序后， 打开浏览器输入地址：<a href="http://localhost:33333/file-upload/upload.html">http://localhost:33333/file-upload/upload.html</a> 可访问大文件上传示例页面。</p>
<h2><a href="#小结" name="小结" class="anchor"><span class="anchor-link"></span></a>小结</h2>
<p>本文以Akka HTTP和HTML 5讲述了怎样实现一个支持大文件断点上传、下载和秒传的示例应用程序。</p>
</div>
<div>
<a href="https://github.com/yangbajing/scala-web-development/tree/master/book/src/main/paradox/routing-dsl/file-upload.md" title="Edit this page" class="md-source-file md-edit">
Edit this page
</a>
</div>
<div class="print-only">
<span class="md-source-file md-version">
1.0.0
</span>
</div>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-nav">
<nav class="md-footer-nav__inner md-grid">
<a href="../routing-dsl/exception.html" title="异常处理" class="md-flex md-footer-nav__link md-footer-nav__link--prev" rel="prev">
<div class="md-flex__cell md-flex__cell--shrink">
<i class="md-icon md-icon--arrow-back md-footer-nav__button"></i>
</div>
<div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title">
<span class="md-flex__ellipsis">
<span class="md-footer-nav__direction">
Previous
</span>
异常处理
</span>
</div>
</a>
<a href="../directives/index.html" title="常用指令" class="md-flex md-footer-nav__link md-footer-nav__link--next" rel="next">
<div class="md-flex__cell md-flex__cell--stretch md-footer-nav__title">
<span class="md-flex__ellipsis">
<span class="md-footer-nav__direction">
Next
</span>
常用指令
</span>
</div>
<div class="md-flex__cell md-flex__cell--shrink">
<i class="md-icon md-icon--arrow-forward md-footer-nav__button"></i>
</div>
</a>
</nav>
</div>
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-footer-copyright">
Powered by
<a href="https://github.com/lightbend/paradox">Paradox</a>
and
<a href="https://jonas.github.io/paradox-material-theme/">Paradox Material Theme</a>

</div>
<div class="md-footer-social">
<a href="https://github.com/yangbajing" class="md-footer-social__link fa fa-github"></a><a href="https://weibo.com/yangbajing" class="md-footer-social__link fa fa-globe"></a><a href="https://www.yangbajing.me/" class="md-footer-social__link fa fa-globe"></a>
</div>

</div>
</div>
</footer>

</div>
<script src="../assets/javascripts/application.583bbe55.js"></script>
<script src="../assets/javascripts/paradox-material-theme.js"></script>
<script>app.initialize({version:"0.17",url:{base:"../."}})</script>
<script type="text/javascript" src="../lib/prettify/prettify.js"></script>
<script type="text/javascript" src="../lib/prettify/lang-scala.js"></script>
<script type="text/javascript">
document.addEventListener("DOMContentLoaded", function(event) {
window.prettyPrint && prettyPrint();
});
</script>
</body>
</html>
